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THE PERIOD FOR RESPONSE: [check only a) or b)] 

a) gg expires 3 months from the mailing date of the final rejection. 

b) n expires either three months from the mailing date of the final rejection, or on the mailing date of this Advisory Action, whichever 

is later. In no event, however, will the statutory period for the response expire later than six months from the date of the final 
rejection. 

Any extension of time must be obtained by filing a petition under 37 CFR 1 .136(a), the proposed response and the appropriate fee. The 
date on which the response, the petition, and the fee have been filed is the date of the response and also the date for the purposes of 
determining the period of extension and the corresponding amount of the fee. Any extension fee pursuant to 37 CFR 1 .17 will be 
calculated from the date of the originally set shortened statutory period for response or as set forth in b) above. 



□ Appellant's Brief is due two months from the date of the Notice of Appeal filed on , 



(or within any 



period for response set forth above, whichever is later). See 37 CFR 1.191(d) and 37 CFR 1.192(a). 

Applicant's response to the final rejection, filed on Apr 12. 2001 has been considered with the following effect, 

but is NOT deemed to place the application in condition for allowance: 

(3 The proposed amendment(s): 

B will be entered upon filing of a Notice of Appeal and an Appeal Brief. 
□ will not be entered because: 

□ they raise new issues that would require further consideration and/or search. (See note below). 

□ they raise the issue of new matter. (See note below). 

Eg they are not deemed to place the application in better form for appeal by materially reducing or simplifying the 
issues for appeal. 

□ they present additional claims without cancelling a corresponding number of finally rejected claims. 

NOTE: Proposed amendment comprising further limiting: "one or more annotat ions sets" to "one or more annotation 

sets of a plurality of annotation sets" does not place the app lication in condition for allowance. 



□ Applicant's response has overcome the following rejection(s): 



□ Newly proposed or amended claims . 



would be allowable if submitted in a 



separate, timely filed amendment cancelling the non-allowable claims. 

B The affidavit, exhibit or request for reconsideration has been considered but does NOT place the application in condition 
for allowance because: 

(see attache d supplemental advisory action) __ 

□ The affidavit or exhibit will NOT be considered because it is not directed SOLELY to issues which were newly raised by the 
Examiner in the final rejection. 

B For purposes of Appeal, the status of the claims is as follows (see attached written explanation, if any): 

Claims allowed: None _ 

Claims objected to: None m . 

Claims rejected: 1-6. 8-25. 27-33. and 35-39 _ 



□ The proposed drawing correction filed on . 



□ has Dhas not been approved by the Examiner. 



□ Note the attached Information Disclosure Statement(s), PTO-1449, Paper No(s). 

□ Other 
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Supplemental Advisory Action 



1. This communication is in response to After Final Amendment/Request for 
reconsideration, filed 04/12/01, wherein proposed amendment further limited claims 8, 
31, and 39, claims 1-6, 8-25, 26-33 and 35-39 remain pending. 

2. Argument present have been fully reconsidered and major point presented are 
addressed hereby : 

It is argued point (A): prior art of record, Sidana does not teach or suggest claim 
(1) limitation: i.e. generation of new annotations based on electronic mail messages 
received not in response to the sent electronic mail messages. 

It is argued point (B): prior art of record, Sidana in combination with Hou do not 
teach or suggest claim (13) limitation: i.e. an installation option that identifies an 
installation program that can be used to install one or more modules allowing the content 
and corresponding media content to be rendered; 

It is argued pint (C): prior art of record, Hou does not teach or suggest claim 
limitation: i.e. an identifier of a temporal range of the media content that the new 
annotation content is associated with, which corresponds to a location of the media 
content that is after the beginning of the media content and at which rendering of the 
media content should begin in response to selection of the identifier of the media content 
in the electronic mail message. 

3 . In response to point A: Hou teaches; 

(a) By utilizing the present invention, an individual can perform the following tasks: (1) an 
individual can create reports by i ncluding multiple media data ; place multimedia data objects into the 
canvas (region on the display provided to perform media-related actions) and the individual can send the 
report to other individuals via an e-mail system ; (i.e. generation of an e-mail message containing 
annotation, where generated e-mail message is not in response to a sent message)', (col 3/lines 8-13); The 
open report handler receives an input from the command interpreter, and a decision is made as to whether 
the report is a new report and if there is a new report the current report is cleared , (i.e. generation of new 
report, where report is not a stored report which may be from the mailbox); (col 3/lines 65-col 4/line 14, 
Fig. 2, new report or from mailbox); the mail report handler receives an input..., the mailing lis t is 
searched and displayed, the user selects a receiver's name , a report is composed to a message and the 
message is delivered (col 4/lines 26-31, Fig. 4 from Fig. 1); The attach media handler (26) attaches media 
types of text, images, graphics and audio from an external media source 31. into the multimedia report 
(32) . ...the mail rep ort handler (27) connected to the multimedia report 32 generates an e-mail message 
for delivery (col 6/lines 4-25); (i.e. generation of an e-mail message containing annotation, where 
generated e-mail message is not in response to a sent message) 

4. In response to point B: Russell teaches; 

The operating system sub-programs 171 consist of the Windows 3.1 operating system, the 
multimedia extensions which come as part of the retail package containing the operating system , and the 
device drivers selectively loaded when the PC is configured, (i.e. an installation option that identifies an 
installation program, (option, the act, power or freedom of choosing; choice, something available as a 
choice, the exclusive right, usually obtained for a fee, to buy or sell something within a specified time at a 
set price, c. A right to buy or sell specific securities or commodities at a stated price within a specified 
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time)). Said program extensions (Fig. 17) included in said device drivers (e.g. the sound board driver, and 
the drivers for display ), further including a second group of subprograms (DLL) and three executable 
application subprograms., specifically these executable subprograms comprise the items App.exe 177, 
Record.exe 179. and Buffer.exe 181 , wherein Record.exe and Buffer.exe direct the speech process part 
137 of Fig.. 15, and App.exe 177 directs the User Interface Part 139 of Fig. 15. When the user wishes to 
have the application active ...he starts the application Record.exe 177 Record.exe 177 in turn starts 
Buffer.exe 181. loading the subprograms (DLL) . Record.exe manages the interface to the multimedia 
extensions using the Low-level audio functions, . . . opens the audio device represented by the sound board 
manages the memory used for recording (e.g. multimedia RIFF chunks) by passing buffers to the opened 
device .... ;(col 16/lines 47-col 17/line 40); ...the user has the freedom to recognize part way into a 
telephone call that valuable information has been exchanged, he may at this time invoke the sub-program 
App.exe to actually create a representation of the current and past speech which he can then act on. the 
user has time to hear and evaluate speech, and he has the visual cues to mark and to save the speech after 
he has heard it (i.e. program extension (loadable modules) allows media content to be rendered). App.exe 
..create specialized windows, timers, and file management systems, ... the operation of A pp.exe is 
governed by the two timers, and bv user events generalized in FIG. 18 as keyboard events 193 and 
mouse events 195; (col 18/lines 25-56, Figs, 6-18, rendering content and multimedia content, also 
comprising identifier's of temporal range of media content)', 



5. In response to point C: Hou teaches 

In the report viewer, 35 of FIG. 9, the dynamic annotation can be displayed along a time-line with 
some markers as illustrated in FIG. 14. These markers are used to specify annotation segments for 
playing back or for reply. Four types of markers are used: object-region markers . ... and voice-silence 
markers. Object-region marker s indicate which data object the mouse pointer is pointing at along the time 
line . For the purpose of visual display, these markers divide the time line into several regions , each 
representing one data object that the mouse pointer is located. Furthermore, each region is color-coded to 
distinguish different data objects . ... The silence parts of the human voice are used as voice-silence 
markers to indicate which portions of the annotation have human voice (col 8/lines 43-67); (an identifier 
of a temporal range of the media content that the new annotation content is associated with, which 
corresponds to a location of the media content that is after the beginning of the media content and at 
which rendering of the media content should begin in response to selection of the identifier of the media 
content in the electronic mail message), ...The content of the reports goes beyond other available 
reporting systems by having dynamic annotation capability . . . . the report can also serve as a consultation 
tool for other individuals to add their comments bv using dynamic annotations . The segmentation of 
annotations and display of related annotations enable the individual to identify related annotations , (col 
9/lines 14-18). A multimedia-based reporting system... comprising: display means for displaying 
segmented annotations using markers : user selection means to allow said user to select a segment; record 
means to record said segment; and, playback means to playback said segment, (col 9/lines 64-col 10/line 
2); (an identifier of a temporal range of the media content that the new annotation content is associated 
with, which corresponds to a location of the media content that is after the beginning of the media content 
and at which rendering of the media content should begin in response to selection of the identifier of the 
media content in the electronic mail message) 



6. 



Arguments filed 04/12/01 have been fully considered, but not found persuasive. 
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Conclusion 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Prieto, B. whose telephone number is (703) 305-0750. 
The Examiner can normally be reached on Monday-Friday from 6:30 to 4:00 p.m. If 
attempts to reach the examiner by telephone are unsuccessful, the Examiner's Supervisor, 
Mark H. Rinehart can be reached on (703) 305-4815. The fax phone number for the 
organization where this application or proceeding is assigned is (703) 308-6606. Any 
inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is (703) 305-3800/4700. 

Any response to this action should be mailed to: 

Commissioner of Patents and Trademarks 
Washington, D.C. 20231 

or faxed to: 

(703) 308-9051, (for formal communications intended for entry) 

Or: 

(703) 305-7201 (for informal or draft communications, please label 
"PROPOSED" or "DRAFT") 

Hand-delivered responses should be brought to Crystal Park II, 2121 Crystal Drive, 
Arlington. VA, Fourth Floor (Receptionist), further ensuring that a receipt is provided 
stamped "TC2100". 



B. Prieto 
Patent Examiner 
April 14, 2001 
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